Integrated tool for compliance testing within an enterprise content management system

ABSTRACT

Methods, tools, systems and computer readable media for compliance testing instrumentation and/or software. Data from one or more analytical instruments and/or software is converted to a technology-neutral format, which is independent of instrument type, instrument model, instrument manufacturer and data type of the analytical instrument or software from which the data was outputted. Calculations are performed on the converted data to produce one or more outputs, and then selection is made from the one or more outputs to populate a final report, wherein the one or more outputs are standardized and are directly comparable to outputs resultant from carrying out the method on another set of one or more other analytical instruments and/or software, irrespective of manufacturer or model of the other analytical instruments and/or software.

BACKGROUND OF THE INVENTION

Qualification of instruments for regulated markets has traditionallyfollowed one of two models: paper-based protocols that are run oninstruments using the native controllers of the respective instruments;and external calculations or qualification routines that are embeddedinto the controlling softwares of the instruments, respectively. Someefforts at automated data collection have required that an alternativedata path be employed for the data collection while still controllingthe instrument suing its native controller. Examples of proprietaryembedded software suites include Cerity NDS (Agilent Technologies, Inc.,Palo Alto, Calif. for chemical/pharmaceutical quality assurance andquality control, and Empower CDS (based on Waters Millenium software,Waters, Inc., Milford, Mass. These software suites are limited to thesuite of instruments that they can control.

Thus, even though these suites are proficient for producing data andresults for qualification/quality assurance tasks for the particularinstruments that they control, such suites cannot providestandardization of the data types that are processed, nor calculationsacross controller platforms. Further, data is maintained in aproprietary format that requires the collecting data system to bepresent and functioning for viewing and reprocessing any dataused/outputted by these systems.

Recently, an integrated single source of data collection and storage wasintroduced by Scientific Software (e.g., EZChrom Elite). While offeringa relatively large driver set, this solution is still limited by theavailable drivers that are provided with the solution.

Further, all of the current solutions, including those mentioned above,as applied to instrument qualification, require decoupling of the nativesystem that controls the instrument to be qualified, in one fashion oranother.

It would be desirable to provide a solution capable of incorporatingdata from different instruments, as well as from differentmanufacturers, to compile reports thereon. It would be further desirablethat such a solution provides standardization among various data typesso that one platform can be readily used to generate reports using datagenerated from instrument having different platforms, and/or still otherinstruments that aren't included with any established platforms.Accordingly, there is a need for solutions that are generally applicablefor use with data generated/collected by instruments from most, if notall manufacturers, to readily prepare reports therefrom and/or otherwisemanipulate the data as needed

SUMMARY OF THE INVENTION

Methods, tools, systems and computer readable media for compliancetesting of analytical information are provided. Data outputted from atleast one analytical instrument and/or software may be converted to atechnology-neutral, format, which is independent of instrument type,instrument model, instrument manufacturer and data type of theanalytical instrument from which the data was outputted. One or morecalculations are then carried out on the converted data to produce oneor more outputs, and selection from the one or more outputs is made topopulate a final report. The one or more outputs are standardized andare directly comparable to outputs resultant from the method beingcarried out on data from another set of one or more other analyticalinstruments, irrespective of manufacturer or model of the otheranalytical instruments.

Systems, tools, methods and computer readable media are provided forstandardizing characterizations of at least one of analytical hardwareand controlling software during compliance testing. Algorithms areprovided for converting data from a native format as outputted by ananalytical or other instrument to a technology-neutral format. A datareduction engine is configured to reduce the data to atechnology-independent, reduced metadata set. A calculation engine isconfigured to perform at least one calculation on at least one of thedata converted to a technology-neutral, standardized format and thereduced metadata set to produce one or more outputs required for a setof predefined criteria.

Systems, tools, methods and computer readable media are provided forcompliance testing at least one of instrumentation and software,including: displaying a test protocol on a user interface and promptinga user to input information regarding results of a test; prompting atleast one instrument or software associated with an instrument toinitiate the test protocol in response to an input by the user into thetest protocol displayed on the user interface, or results from anotherinstrument in response to a test protocol run on the another instrument;and automatically calculating results of the test protocol run on the atleast one instrument.

These and other advantages and features of the invention will becomeapparent to those persons skilled in the art upon reading the details ofthe system, tools, methods and computer readable media as more fullydescribed below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flowchart of events that may be carried out duringprocessing according to at least one embodiment of the presentinvention.

FIG. 2 illustrates one example of a form that may be used by anembodiment of the present invention.

FIG. 3 is a schematic flow chart illustrating process flow according toan embodiment of the present invention.

FIG. 4 is a block diagram illustrating functions and processes that maybe managed through a business process manager according to an embodimentof the present invention.

FIG. 5 illustrates an extraction of information from a form.

FIG. 6 is a flow chart illustrating further details of process flow byan embodiment of the present invention.

FIG. 7 is a schematic representation of an embodiment of a system foruse in creating a compliance report for chromatographic instrumentation.

DETAILED DESCRIPTION OF THE INVENTION

Before the present systems, methods and computer readable media aredescribed, it is to be understood that this invention is not limited toparticular hardware, software, or media described, as such may, ofcourse, vary. It is also to be understood that the terminology usedherein is for the purpose of describing particular embodiments only, andis not intended to be limiting, since the scope of the present inventionwill be limited only by the appended claims.

Where a range of values is provided, it is understood that eachintervening value, to the tenth of the unit of the lower limit unlessthe context clearly dictates otherwise, between the upper and lowerlimits of that range is also specifically disclosed. Each smaller rangebetween any stated value or intervening value in a stated range and anyother stated or intervening value in that stated range is encompassedwithin the invention. The upper and lower limits of these smaller rangesmay independently be included or excluded in the range, and each rangewhere either, neither or both limits are included in the smaller rangesis also encompassed within the invention, subject to any specificallyexcluded limit in the stated range. Where the stated range includes oneor both of the limits, ranges excluding either or both of those includedlimits are also included in the invention.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although any methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of the present invention, the preferred methodsand materials are now described. All publications mentioned herein areincorporated herein by reference to disclose and describe the methodsand/or materials in connection with which the publications are cited.

It must be noted that as used herein and in the appended claims, thesingular forms “a”, “and”, and “the” include plural referents unless thecontext clearly dictates otherwise. Thus, for example, reference to “aform” includes a plurality of such forms and reference to “theanalytical instrument” includes reference to one or more analyticalinstruments and equivalents thereof known to those skilled in the art,and so forth.

The publications discussed herein are provided solely for theirdisclosure prior to the filing date of the present application. Nothingherein is to be construed as an admission that the present invention isnot entitled to antedate such publication by virtue of prior invention.Further, the dates of publication provided may be different from theactual publication dates which may need to be independently confirmed.

DEFINITIONS

A “platform” as used herein refers to a support infrastructure foracceptance and coordination of tools and applications required toperform a series of related, but diverse tasks.

An “enterprise content manager” refers to a system, scalable toenterprise levels, composed of various hardware and software elementsthat support the secure collection, indexing and storage of electronicobjects.

Disclosed herein are methods, systems and computer readable media forprocessing data outputted by analytical instruments in a standardizedmanner so that results of processing are directly comparable withresults from processing data outputted by other instruments, regardlessof model or manufacturer. Methods, tools and computer readable media forgenerating, transmitting and storing forms specific for a user's needsare provided, including, but not limited to compliance validation forms.Systems include computers and associated hardware that may beconnectable to a network (for internet or intranet use) that can executerules for a selected form (e.g., one non-limiting example is a formsuitable for submission to a regulatory agency such as the FDA). In oneembodiment, a system is provided to perform analytical hardwarequalifications.

Calculations may be performed to answer a series of questions relatingto one or more performance tests designed to determine compliance of ananalytical instrument and/or software under consideration with a set ofpredefined criteria. Such predefined criteria may be criteria definedfor regulated industries. For example, “predefined criteria” include,but are not limited to regulations set forth in the Food, Drug, andCosmetic Act. Predefined criteria are limits and criteria that representbest practices and manufacturers specifications relating to instrumentoperation and performance. Compliance to these acceptance criteriaprovides documented evidence of a device's operation within expectationof intended use. Such compliance is required by law and is listed in theCode of Federal Regulations under headings Part 210, 211, 820, 58, and21-Part 11 as well as other such regulations and guidance as applies.

Forms may be used as built-in records to store data as it occurs,lending to use of the forms for tracking/audit trails. The forms arefurther useable as a basis for generating reports in a variety offormats. However, as reports are changed, the underlying processes(e.g., the forms containing the data from which the reports aregenerated) stay the same. Basic universal forms stay the same, while thedata they contain can be used to report in many different ways. Theforms may be provided to a user in a “wizard-type” environment (i.e., asa “wizard-type” interface), wherein the user is prompted for simpletasks, the response to which are incorporated into a much larger dataproduct. In this way the user only has to deal with simple, single itemtasks, one-at-a time.

An enterprise content manager (ECM) may be employed to provide a secureplatform to manage all data storage, metadata extraction and archival ofdata. One non-limiting example of an ECM that may be employed is aCerity ECM, e.g., available from Agilent Technologies, Inc., Palo Alto,Calif. Since an ECM is an enterprise system, it also providesscalability to the present system.

Referring to FIG. 1, data is inputted to the system in its native formatat event 110, for initial conversion to a technology neutral format(event 120) so that all further processing is with respect to data thatis all in the same format, regardless of which instrument was used tooriginally output the data in its native format. Data reduction may beperformed at event 125 by a data reduction engine as described below, ifneeded. By performing calculations/further processing (event 130) ontechnology neutral formatted data with the present system, calculationsare thereby standardized, so that results (event 140) are directlycomparable between data produced by various instruments, models andmanufacturers. The standardization is made possible by the system'sability to convert data from external sources into a technology neutralformat which is then data reduced and calculated by common components.Further, this standardization may be applied to data sources manually,semi-automatically (requiring some manual application) or automatically,and such data may require data reduction, or may be in a ready toprocess form. In this way data that characterizes the operation ofinstrumentation or controlling data systems can be used for the purposesof qualification of said device irrespective of proprietary or nativeformat. One non-limiting example of a technology neutral format that maybe employed by the present system is referred to as AnalyticalInformation Markup Language (AnIML,) which is an open source, XML-basedstandard for formatting analytical data. By converting all data to atechnology neutral format, and then processing the converted data allaccording to the same protocols, results are generated that arestandardized and directly comparable among results for differentinstruments which may be different models and/or made by differentmanufacturers.

As noted, the computerized data system (CDS) that is in place foroperating the instrument(s) to obtain the data on which a report is tobe generated may be used as direct input to the system. Thus, originaldata collected for a report may be accomplished using the nativecontrolling software (of the CDS) of the instrument(s) without the needto go through external analog to digital conversion or othermanipulation. Original data, which may be preserved for possiblereanalysis by the native CDS, is also converted to an acceptedtechnology-neutral format allowing the data to be submitted to a singlereprocess and calculation engine for consistent reduction andprocessing. By using the native CDS, the present system may also makeuse of the drivers employed by the native CDS, thereby furtherfacilitating the universal applicability of the present system todifferent types of instruments and to instruments having differentstandards/CDS's as a result of being produced by differentmanufacturers.

Instructions may be instantiated as forms 200 (e.g., see FIG. 2) toprovide procedural information, while also functioning as datarepositories. Forms 200 may be constructed in many different ways andpresented to have as many different appearances, some of which aredictated by the information to be displayed/stored and much of which maybe flexibly designed. The instrument/process type as well as therequired input to the form 200 dictates the content and appearance ofform 200. FIG. 2 shows one example of a form 200 which is in no waymeant to be limiting, as many different forms may be provided by thesystem. In FIG. 2, form 200 includes fields instructing the followingdata to be inserted and stored: Instrument Name 202; Other Name 204;Channel Description: Split-Splitless 206, Purged-Packed 208, VolatilesInlet 210; Model Number 212; Serial Number 214; and License CertificateID 216. Forms 200 may be run as an applications program interface (API)and, as such need not ever be even visualized by a user of the systemwhen all data fields can be automatically identified from the technologyneutral formatted data and/or native CDS and populated into form 200 tocomplete all data fields.

Alternatively, or additionally, a user interface 250 may be provided todisplay one or more forms. In a case where automatic population of allrequired data is not possible, user interface 250 can display form 200so that a user can interactively select an entry (as in the case ofModel Number 212, shown in FIG. 2, where a drop down menu is providedfrom which the user can selected the proper entry) or manually input anentry, such as by typing, cutting and pasting, scanning or some otheralternative data entry mechanism that requires intervention on the partof the user. Further alternatively, the user interface 250 may beoptionally used to enter all data required by a form, either as a resultof user preference to do so, or because an instrument being consideredis sufficiently old or unsophisticated, so as to lack a sophisticatedenough software interface to supply some or all of the dataautomatically by interfacing with the present system.

As another alternative, a user interface may display a test protocolthat prompts the user to input information regarding results of a test.In some instances, the test may be automated, wherein the system mayprompt one or more lab instruments to initiate a test protocol inresponse to one or more answers inputted into the user interface by theuser in response to questions asked on an interactive form/testprotocol, or in response to results from another instrument (e.g., inresponse to a test protocol designed for that instrument).

The system may also provide a report detailing processes and/orinstruments that do not comply with selected specification (i.e., aprotocol deviation form).

The forms may be XML based forms that can be directly rendered to afinal report (such as in pdf format, or other format suitable for paperdocuments, for example). Thus, for example, forms 200 may be displayedin pdf or some other document format on user interface 250 when part orall of them are to be interactively filled out by a user. As noted, partor all of forms 200 may be programmatically filled out from autodetection of calculation engines provided by the system. Forms 200 maybe left in native XML format and thereby function as storage for thedata that they contain. Forms 200 may be further rendered from the XMLformat to an HTML version for use with a browser.

When used interactively, forms may be presented to a user according toneed and thus, forms that apply only to the instrument(s) under test arepresented, thereby reducing delivery complexity and error potential,while at the same time providing audit trails for tracking, since theforms may be saved, as noted above. By converting proprietary data intostandardized data (i.e., data having a technology neutral format), thesystem may provide data in a standardized output form. Thus,inconsistent output from instruments can be converted to consistentinput to an engine that can do calculations in a very predictable,standardized way, which is an important consideration for qualificationand compliance reports.

Once native data has been converted into technology neutrally formatteddata, metadata may be created by data reduction engine 302 of the system100 so that algorithms from the instrument's 10 system(s) do not need tobe relied upon, and this further ensures standardization of results. Forexample, for application to chromatography, the present system does notrely upon the software 12 running the chromatography instrument 10 fromwhich the raw data is generated to determine what is a peak in the dataor where to define the location of that peak, as such determinations aremade based upon calculations and algorithms run by the data reductionengine 302 of the present system. Data reduction engine 302 reads thedata having been converted into technology neutrally formatted data andconverts this digital representation of an analog function into datarepresenting features described/characterized by the data (e.g., peaks,noise, gradient steps, etc.). The same applies to other calculations,such as those determining and or filtering noise levels, etc. Using thisapproach, consistent results are determined for data across the board,whether a particular type of instrument 10 was manufactured by one oranother particular manufacturer, or whether the instrument 10 is adifferent model than another, both of which data is being processedfrom.

As one example, signal data from a chromatography instrument 612, asinputted to system 100 by the native controlling software for theinstrument is just a series of changing signals over time. Reductionengine 302 converts these signals (having been converted to a technologyneutral format) into useable data, e.g., peak area, noise calculations,etc. —which can be fed to calculation engine 306—e.g., there are 5 peaksand those 5 peaks have peak areas of 2, 2.1, 1.9, 2 and 2 and the meanis X, with standard deviation of Y, etc. so that these values can becompared to an acceptance standard, or with like values calculated withregard to another instrument 612.

Depending upon the instrument that has generated the data, a datareduction engine 302 may not even be needed. For example, a balancealready outputs data that is reduced to numbers that are useable bycalculation engine 306 and so this data doe not need to be furtherreduced, although it may need to be converted to a technology neutralformat. Further, other alternative reduction engines 302 may be includedwith the system 100 as part of a library that may be accessed fornon-standard reduction requirements. By performing data reduction with acomponent of system 100, this separates reliability on each instrument'ssoftware for performing such functions. Accordingly, all data reductionis standardized across reports that are prepared by system 100, andperformance is all standardized by evaluation by the same system.

Further, since the data is standardized, only one method need bedeveloped to produce a particular type of report based on the data, asopposed to the current need to create a method for each instrument thatemploys a different data type or format. Thus, calculation engine 306can perform calculations based upon a single library 304 (e.g., seriesof calculations tailored to a specific type of report for a particulartype of data reporting). That is, since consistent raw data sets (i.e.,technology neutrally formatted data) are received by data reductionengine 302, these data set can be properly manipulated with a singleconsistent method. Thus although the method for acquisition of data mayvary depending upon the computer data system from which the data isbeing acquired, once that data has been converted to a technologyneutral format, the back end processing is consistent (e.g., processingby data reduction and calculation engines, etc.). Library 304 typicallycontains a set of calculations for performance of the standardized tasksin the back end processing (e.g., calculation/identification of peaks;calculation of statistics describing the data, etc.). With respect todata reduction and calculation, the results may be standardized andindependent of the originating data-system or controlled instrument, asnoted above. Reports based on those results are fully customizable, asreports ranging from simple summary reports to traditional, fullydescribed compliance protocols may be outputted.

The library can be modified, typically added to, to increasefunctionality, but it does not have to be a different library based onthe data system that the instrument used, contrary to what is currentlyrequired. Consequently, calls become consistent and calculations becomereusable and portable. For example, a library may be created tocalculate peak precision, signal-to-noise, etc., and library 304 may bebuilt to accept only consistent input forms because the input formatwill always be the same, since the engine for extracting data (datareduction engine 302) will always be the same. Running processes in thisway provides consistent metrics across all manufacturers, types andmodels of instrumentation. For example, peak detection and baselineevaluation can be performed as de facto standards against which allsystems/instruments may be evaluated. Thus, such a library 304 isreusable and portable, being applicable to calculation of the defineddata specifications based upon data inputted from the data reductionengine 302, and wherein data reduction engine 302 may be applied to datafrom any applicable instrument for which it makes sense to calculate theprescribed specifications, since the data from the instrument will havebeen converted to a technology neutral format that the data reductionengine 302 is configured to receive as input.

The standardization of processing will advantageously reduce trainingrequirements for operating personnel, since personnel will no longerneed to be trained for operating with regard to each different piece ofequipment, but can instead be trained to run the standardized processes.For example, under conditions prior to the present invention, it wouldnot be unusual for an instrument (piece of hardware) 10 to be operatingin various locations under multiple (e.g., three) different proprietaryoperating software platforms. For compliance purposes, it might then benecessary to replicate the compliance procedures as many times as thereare multiple platforms. By providing the present system as built on anindependent platform, it is not dependent upon the operating software ofthe particular instrument upon which reports are to be generated. Inthis way the system is readily adaptable to new/various hardwares aswell as softwares, given the generic nature of the protocols.

As noted, system 100 may further employ a calculation engine 306 toperform calculations on the reduced metadata set produced by datareduction engine 302 for formulating standardized results 308.Calculation engine 306 performs calculations on metadata in the reducedmetadata set received from data reduction engine 302 as well as anycalculations that may need to be performed on other data that has beenconverted to the technology-neutral format, as instructed by forms 200,such as for the performance of qualification services on analyticalinstruments as well as other instruments. As noted, forms 200 may act asinstructions for processes carried out by calculation engine, as well asfor data storage repositories of the results of these calculations.Forms 200 can contain any combination of input types includinginteractive manual input, information detected by software of system 100and/or the CDS of the instrument being considered and/or calculatedreduced data. Forms 200 may further include launch points forexecutables that perform detection, calculation, or any other functioncalled for by the process. Forms 200 may be version controlled andstored as record of the data collection process leading to a resultingfinal report. In this way the stored versions of forms can serve as anaudit trail from the time of initial collection of the data all the waythrough to the time of the issuance of the final report.

Because of the standardization of all data to a technology neutralformat and creation of metadata from the same, calculations bycalculation engine may be carried out by calls to a consistent andtested library, as the calculation are reusable and portable.

In order to manage data storage, metadata extraction and archival ofdata, as well as compilation of final reports and other form managementfunctions, system 100 may employ an enterprise content manager (ECM)404, as noted earlier. ECM 404 may provide a secure platform on which tomanage the functions described. FIG. 4 illustrates a flowchart offunctions and processes that may be managed by ECM 404 via businessprocess manager (BPM) 406. BPM 406 manages flow so that data storage andformat conversion (to a technology neutral format) are carried out byECM 404 at event 408, followed by reprocessing/data reduction by datareduction engine at event 410, functions of which were described above.Further calculations are carried out by calculation engine 306 at event412, which may be based upon instructions contained in forms 200 and thedata populated into form 200 may be recorded and stored in ECM 404 atevent 414. The record forms 200 may then be data mined at event 416 byrecord mining engine 440 to extract specific items of data/metadata thatare required to populate a final report.

FIG. 5 illustrates data extraction from a form 200 to obtain informationneeded for preparing a report, wherein a portion of a record form 200 isshown from which a particular data entry 502 is located. Record miningengine 440 may employ toolsets for mining data, e.g., name-value pairsmay be taken from forms 200 and calculation engine 306 may furtherextract those values needed by identifying such values based upon thenames associated with the values in the name-value pairs. Data from aform 200 can be calculated and the resulting calculations may bereturned to the same form 200 or to another form 200 as needed forpurposes of organization, readability, clarity, etc.

As shown, forms 200 actually do contain the information/data receivedfrom the software of the instrument being considered, and that data canbe mined to fill out automated report applications or otherwise to fillout a final report 444. In this way, forms 200 act as a repository thatcan be mined in various ways—compliance, asset management, etc. Once afinal report 444 is signed, however, the data that was mined to fill outthe final report document 444 can no longer be changed, ensuringinviolable metadata, so that an effective audit trail is maintained.

An automated report application (automated report generator) 442 may beoptionally included, and if used, functions to automatically populatedocuments at event 418 which are then outputted as a customizable report444 at event 420. Automated report generator 442 is an application thatfacilitates that construction of configuration-specific documents from alibrary of all possible configurations. Automated report generator 442allows documents to be populated with content learned through manyvarious mechanisms, such as the mechanism that have populated forms 200.An analogy to one function of the automate report application 442 iswith reference to an automobile and an automobile user's manual thataccompanies the automobile. Because the owner will typically have manyoptions from which to choose from, the owner's manual is typicallywritten to describe each of these options. Thus, for example, if anowner has a particular type of sound system, but there are six differentsound system options for the owner's car, in order to access informationabout the sound system, the owner will typically have to page throughdescriptions of all six sound systems options in the user's manual untilthe matching sound system is found. An automated report application forowner's manuals would prepare this user's manual based upon the optionschosen by the owner, so that when the owner looked up the description ofthe sound system, only one sound system would be described in theowner's manual, i.e., the description for the sound system actuallyselected by the owner for his/her car.

The automatically populated forms 200 as well as the final report 444may be stored into ECM 404 so that ECM 404 is the location of theinitial collection, calculation, meta-data and final data, as well asaudit trails. Thus, system 100 may include a relational database withtools such as data reduction engine 302, calculation engine 306, andrecord mining engine 440, for example, sitting on top of it.

Reports 444 can take on any form, and may be selected by a user. Forexample, a report may be created in summary form or in full detail, withor without a logo, etc. While the reports 444 are customizable, theunderlying forms 200 created by the system 100 do not change so thatstandardization is preserved. Auto-documentation feature 442 may beoptionally provided, as noted above, whereby the user is provided withselectable choices, via user interface 250, to determine the format ofthe report 444 to be produced. Thus, depending upon the selection made,different groupings of metadata from the underlying forms 200 areselected and combined into a format of the final form selected.

Audit trails may be automatically provided by the metadata stored asforms 200 by ECM 404, as alluded to above. Further BPM 406 may providee-mail notification, scheduling and review, electronic signaturefunctions, etc.

Referring now to FIG. 6, a flow chart that further explains process flowby the system is shown. As noted above, a user interface may or may notbe needed, which may depend upon the choice of the user, the types ofinstruments being reported upon, and/or whether the system is capable offully automatically obtaining all information required to generate afinal report. System control API's 602 are provided for runningprocesses so that there is not a need to display the process on userinterface 250. The business process manager (BPM) permits flexibleformatting of process. For example, the process can be changed just bychanging/rearranging a flow chart similar to that which is shown in FIG.6. For example, flow charts used may be flow charts produced byMicrosoft Visio (Microsoft Corporation. Redmond. Washington) or otheralternative chart building software that allows flow charts to bereadily modified interactively. Such chart building softwares provide avisual manifestation of a process implemented and controlled by BPM 406.As a simple example, if a current process flow of system 100 includes aprocess or subprocess defined by steps A>>B>>C>>, but the currentuser/client requires step B to be performed after step A and before stepC, then the current process chart can be interactively rearranged, suchas by dragging step C between steps A and B and dropping it there toresult in a process/subprocess defined by the steps A>>C>>B.Accordingly, the system 100 provides a great amount of flexibility forcustomizing the process control, which is then managed by BPM 406 usingforms-based process management 604 as was described earlier.

The technology neutral design of system 100 allows any client's ormanufacturer's data system (i.e., Instrument's Data System 606) to befed into ECM 404. Accordingly, any type of instrument, model ofinstrument or manufacturer of an instrument may be included asinstruments 612 from which data can be received by system 100. Forexample, Instrument 1 may be a liquid chromatography/gas chromatographyinstrument 612 produced by a first manufacturer, Instrument 2 may be aliquid chromatography/gas chromatography instrument 612 produced by asecond manufacturer, Instrument 3 may be still another liquidchromatography/gas chromatography instrument 612 produced by a thirdmanufacturer, Instrument 4 may be a mixed vendor system, Instrument 5may be a refrigerator with an embedded microprocessor or otherassociated hardware/software configured to input data to system 100 (oralternatively, data from this instrument may be manually inputted viainterface 250 if Instrument 5 is not sufficiently automated), andInstrument 6 may be a centrifuge, wherein the same considerations applyas described with regard to Instrument 5. A “mixed vendor system” refersto systems produced by more that one manufacturer/vendor. Examples ofmixed vendor systems include, but are not limited to: a computer datasystem manufactured by a first vendor and controlling an instrumentmanufactured by a second vendor; a computer data system produced by afirst vendor that controls instruments produced by second, third andfourth different vendors; or a computer data system produced by a firstvendor and controlling a single instrument made up of componentsproduced by different vendors, etc. As long as the associated computerdata system can successfully drive the mixed vendor system, the presentsystem can process the data in a manner as described.

As noted above, if the instrument's data is proprietary data, theproprietary data is converted to technology neutral formatted data,(e.g., AnIML) using data system control API's 602 (or, if provided inhuman readable form, the data may be added to the forms manually andincluded into qualification processing with any required calculations)and both the proprietary data and the converted, technology neutral datamay be saved in ECM 404.

The technology neutral formatted data can then be further processed bydata reduction engine 302, calculation engine 306 and reporting engine608. Reporting engine 608 requires at least one of a data miningapplication (e.g., record mining engine 440) or a middleware componentconfigured to provide an input file to reporting engine 608 to correctlypopulate a report.

Once final report 444 has been generated, BPM 406 can direct reviews andsignatures electronically at event 610. The final report, both signedand unsigned may be stored in ECM 404. Further, all intermediate forms200 and the data that they store may be stored in ECM 404 to maintain acomplete audit trail, as was also discussed. All processing representedin FIG. 6 may be based on forms and the instructions contained therein.WYSIWYG authoring capability may be provided by the forms designerapplication for designing forms 200. Secure data handling is ensured byECM 404. Standardized results are the end product of these methods,providing a clear differentiator over anything that is currentlyavailable in the market.

The data path that the instrument 612 uses is the same data path thatsystem 100 uses for reports such as compliance. However, thecalculations performed on the data for whatever report is to beproduced, do not need to be performed on the instrument itself, nor doesthe instrument's software need to be employed for performingcalculations. Advantageously system 100 provides everything that isneeded for performing such calculations. This effectively reduces thenative computer data system (CDS) to a controller and data acquirer.Such reduction provides checks on the interplay between the hardware andsoftware of a system to be qualified at each qualification event withoutburdening the hardware qualification event with data reductionevaluation of the native CDS. This assures that the more frequentrequirements for hardware qualification provide the maximum value withrespect to CDS verification, without forcing extensive CDS evaluation.Further, the controlling system (CDS) need not be qualified for use inthe qualifying of hardware, since it is not used for such purpose bysystem 100. Rather, system 100 performs calculations on the raw dataproduced by the instrument (after conversion to a technology neutralformat, if necessary), thereby taking the instrument's controllingsoftware out of the loop and effectively separating the instrument'shardware, from the associated software, so that the report can focus onthe hardware, independent of qualify the instrument's controllingsoftware.

The modularity provided by system 100 facilitates modular instrumentqualification after repair. The process flow manager 406 may presentforms specific to the tests associated with the requalification of amodule. The resultant data can be associated with the module, stack andexisting compliance documentation to requalify the module. Thus, if amodule needs to be repaired, then that module can be requalified, ratherthan having to requalify the entire system, i.e., qualification may bedone on a modular basis.

BPM 406 may control the workflow from collection of data throughapprovals/signatures of final report 444, and may be tightly integratedinto ECM 404. The entirety of processing may be web browser-based orterminal servers-based so that no footprint is imposed upon the user'squalified computer. In instances where ECM 404 has been incorporatedinto a customer's system, local interfaces (e.g., user interface 250)may be employed.

Referring now to FIG. 7, a more specific schematic representation ofsystem 100 is shown for use in creating a compliance report forchromatographic instrumentation. System 100 is represented asinterfacing 702 with native CDS to receive data inputs. In this example,the equipment being reported on is mixed vendor equipment 612, in whichcase, any or all of the vendor's computer data systems 704 may beemployed through which data is inputted to system 100. Typically,however, a common data system controller (CDS) is provided to controlall of the mixed vendor modules, as noted above. Forms 200 that aredriven by BPM 406 may be presented to a user by placement into auser-specific inbox (i.e., BPM Inbox 706), that functions similarly tothe inbox of an e-mail service. In this way, simple instructions can beprovided in a “wizard” like environment (i.e., where simple tasks arecompleted sequentially and interactively). Thus, if a message is placedin inbox 706 that instructs a simple task to be performed, once the taskis performed or “Done”, then the next task can be emailed or placed intoinbox 706. At event 708, a user, or manager assigning tasks to a user,may choose the type of test or qualification to be performed, Inresponse to this choice BPM 406 may then run a template to call thecorrect forms to be completed for the chosen test. Configure stack 710provides a configuration-specific template which determines the requiredtests, forms and instructions to be processed. Forms for Instruction 712are one option for processing, herein these forms 200 associated with aqualification event may contain simple instructions for processing withno data entry potential. Forms for Acquisition Process 714 provideanother option for processing according to forms associated with aqualification event in which forms 200 may describe the setup of thenative data system to perform specific runs and acquire specific datafrom the instrument and/or software to be qualified. Those same forms200 may provide controls for entry (which may be manual and/orautomated) of the results obtained from the processes run with respectto the native computer data system to obtain the specific data. Formsfor Manual Entry 716 are forms 200 in which manual entry may be madedirectly to. Alternatively, entry may be made to these forms 200 via anapplication supplied user interface when required by a system beingtested. Manual data 718 refers to a further embodiment of forms 200 thatmay be created such that form elements are present to allow manual,interactive entry of data from an attendant user. Forms 100 may also beconstructed as a mixed model where some elements of the forms 200 areautomatically filled in when the data is available to the system. Whendata is not available to the system for automatically filling in theforms 200, such data can be interactively filled in (manually) by auser.

Compliance auto detection engine 720 may be an applet very similar tocalculation engine 306 that stores or accesses identifyingcharacteristics regarding various types, manufacturers, etc. ofequipment. So for example, where a form requests a model number andserial number of an instrument 612, rather than requiring a user tomanually enter this information, autodetection engine 720 queries thesoftware 704 associated with the piece of equipment 612 to obtain therequired information and then automatically enters it into the form 200from which the request originated. If autodetection engine 720 isunsuccessful in automatically retrieving some of all of the informationthat was queried for, system 100 leaves the entries for this informationon the applicable forms 200 blank and presents the forms for manualcompletion in addition to the automatic generation (autodetected data722) to whatever extent was possible.

Data storage and format conversion of the inputted data are performed byECM 404 as controlled by BPM 406 in accordance with the instructionscontained in forms 200 selected by BPM 406 for processing the data,wherein forms 200 identify the particular data that is needed. In thisexample, data is converted to AnIML formatting 724 or other common dataform (CDF), such as AIA (Analytical Instrument Association) or ANDI(Analytical Data Interchange) format (typically annotated with .cdfextensions), using Native Data ANIML package 726.

Once converted to technology neutral format, data reduction engine 302,which is also tightly integrated to ECM 404 may perform reprocessing ofthe data in accordance with the needs of the final report to begenerated, as instructed by the forms 200 that need to be filled out andas guided by BPM 406. Reprocessing/data reduction calculations can becan be driven by API, so that no user interface is required (i.e.,No-GUI Reprocess 728). Thus, data can be inputted directly from aninstrument's operating software 704 to instrument 100 where it may beconverted to a technology-neutral format and then fed directly to datareduction engine 302.

The reduced/reprocessed data is forwarded to calculation engine 306 (inthis example, calculations are performed for a compliance report, andengine 306 is referred to as a compliance engine) for furthercalculations that are instructed by forms 200. Calculation engine 306may mine forms 200 that have been populated by the reprocessing by datareduction engine 302, or may obtain data from mining results based onmatching names to name-value pairs as described above, perform theinstructed calculations, and, together with the reprocessed data, outputmetadata 730, which is chromatographic metadata in this example. Thisprocessing may also be API driven, so that all processing may be carriedout in the background, without interrupting a user for interactiveinput.

However, even if all the automation cannot work as intended, (such aswhen an instrument lacks adequate software or other capability forautomatically interacting with system 100, for example) then system 100may launch user interface 250 to accept some interactive input from auser, under guidance of a standard operating procedure, so that the usercan interactively choose information to be filled in. Even thecalculation engine 306 is designed to work as an API, as noted. However,a user interface 250 may also be provided for calculation engine 306 toallow a user to use it as a custom calculator, for example, so that thesame results can be manually calculated, through interface with a user,since the custom calculator uses the same engine 306 that the automatedclient uses.

Any or all of the manual data 718, autodetected data 722 and metadata730 may require some additional manual entry(ies) depending upon theparticular instrument from which data is being obtained. Examples ofmetadata entries that may need to be entered manually include, but arenot limited to results of data collected from a source other than thedata source provided by the native CDS, such as readings from onboardsensors, readings from external measurement devices, etc. Forms 200 thatcontain the manual data 718, autodetected data 722 and metadata 730 aremined for the specific data required by the final report 444 to becreated (such as by using record mining engine 440), and the mined datamay be forwarded to an automated report generator application 442 thatassembles the mined data into an automated report input file 732 whichis forwarded to an unparsed master file 734, from which the automatedreport application renders the final document 440.

Alternatively, an automated report application need not be implemented.For example, final reports may be embodied by completed forms 200without the need to data mine such forms. Further alternatively, a finalreport may be compiled by mined data that is simply assembled andattached to the forms 200 containing metadata. Everything between theraw data (e.g., original data received from an instrument orinstrumentation software) and the final reported values is consideredmetadata. Metadata may be raw data or mined data or a combinationthereof as it is used to populate a form. Some pre-final data mayalready be provided on a form while additional pre-final data may needto be added by the process. The data on the forms 200 can all beconsidered metadata in the sense that it is used to create the finalreport data so it qualifies as data about the final report data.

BPM 406 may then forward the final document, such as via e-mail, forexample, to have the final document (which may be in pdf format, as inthe example shown in FIG. 7) signed. The final report cannot be modifiedby those reviewing it, but must be reprocessed by the system 100 ifchanges are to be made. The process flow for such a rerun orre-evaluation involves returning the process to the step that beginsprocessing the information that is desired to be re-evaluated. However,if this is not done, then any changes will still be captured by ECM 404through its automatic audit trails functionality. Further, BPM 406,together with ECM 404 may track the review process and store records ofthe same to maintain the chain of the audit trail. The final report 440is thus a defensible piece for use in meeting compliance regulations.

Forms 200 provide the basis for processing data by system 100.Wizard-like central data collection may be provided wherein either theautomated client or a user are provided with simple tasks to complete byfilling in the appropriate data, which may require a user to type in,scan in, select, or otherwise enter data, or which may require theautomated client to query the instrument's software for the data whichis then inputted to the form, or to perform calculations on selecttechnology neutral data having been converted from the native datareceived from the software of the instrument, or other processing asinstructed by the particular task presented by the form. In their mostbasic configuration, forms 200 are provided to generate a customerdeliverable, typically a final report containing specifically requestedor required data. Thus, forms 200 with standard defaults may be providedto automatically generate such a final report.

Further, forms 200 stored in ECM 404 may be configured to function toprovide an audit trail (such as by storing versions of the forms as theyare completed, together with data and time stamp, for example). Further,forms 200 may be configured to contain instructions for all processingby system 100. For example, certain forms 200 may contain specificinstructions for calculations to be performed by calculation engine 306.Thus, forms 200 can be interactively filled out by a user through userinterface 250, and/or can be programmatically filled out byautodetection processes or calculation engines.

Various combinations of forms 200, automation and custom reporting mayconstitute a final report by system 100. For example, forms 200 alonemay be interactively filled out by a user to prepare a final report.Using ECM 404 together with forms 200, forms 200 along with the finalreport 444 may be centrally stored and provide an audit trail forsupport of the final product. By adding the automated calculationengines, such as data reduction engine 302, calculation engine 306 andrecords mining engine 440, for example, processing may be fullyautomated to provide a final report, if only according to a defaultedform of the final report 444. Adding the autodoumentation application442 provides further flexibility, whereby a final report 444 can becustomized. Note also, that the modules need not be combined asdescribed, or in the order as described. For example, forms 200 may becombined only with automated report application 442, so that a finalreport 444 generated from manual inputs to forms 200 may be customizedusing the automated report application 442.

Further, a hierarchy of forms 200 may be provided for more efficientcompletion of forms 200 during processing. For example, a master formmay be set up to feed other process forms. A master form generallycontains information that is globally the same with respect to allprocess forms that it feeds. Accordingly, this permits that globalinformation to be filled out only once, after which is automaticallyappears in all of the subordinate forms 200 fed by that master form 200.Different types of master forms 200 may also be created. For example, aqualification master form 200 may contain global information such ascustomer information (address, names, etc.), instruments that aqualification will be covering, and/or acceptance limits for instrumentcategories. An instrument configuration master form 200 may contain anamed configuration mapped t configuration details (e.g., a stack ofinstruments 612) and/or override limits for specific equipment needs. Astack, for example, may include all of one type of instrument, differentvendors' instruments, or any combination of instruments, as thecomplexity of the stack can be programmed into an instrumentconfiguration master form 200. Instrument configuration master forms 200may be limited to only those instruments and vendors that areconfiguration master approved, to prevent a user from arbitrarilyattempting to add an instrument to an instrument configuration masterform for which there is no procedure for processing.

Using the methods and systems described herein, non-vendor specificinstrument qualifications may be processed using a native controllingsoftware of an instrument combined with a technology-neutral,standardized, post-collection data reduction and reporting model. Suchprocesses may be provided by universally applicable, scalable,automated, secure and consistent platform for the development, anddelivery of instrument qualification. Original data collected for thequalification may be accomplished using the native controlling softwarewithout the need to go through external analog to digital conversion orother manipulation. However, the system 100 is not precluded from usingalternative data input methods, including, but not limited to data thathas already been digitized; manual input of data, etc. Original data maybe preserved for possible reanalysis by the native controlling software,and may also be converted to an accepted technology-neutral formatallowing the data to be submitted to a single reprocess and calculationengine for consistent reduction and processing. Instructions may beinstantiated as forms, which may provide procedural information as wellas act as data repositories. Forms may be presented according to needand apply only to the instrument under test to reduce deliverycomplexity and error while providing audit trails for tracking.

While the present invention has been described with reference to thespecific embodiments thereof, it should be understood by those skilledin the art that various changes may be made and equivalents may besubstituted without departing from the true spirit and scope of theinvention. In addition, many modifications may be made to adapt aparticular hardware, software, instrument, module, process, process stepor steps, to the objective, spirit and scope of the present invention.All such modifications are intended to be within the scope of the claimsappended hereto.

1. A method of compliance testing at least one of instrumentation andsoftware, said method comprising the steps of: converting data outputtedfrom at least one analytical instrument or software to atechnology-neutral format, which is independent of instrument type,instrument model, instrument manufacturer and data type of theanalytical instrument or software from which the data was outputted; andperforming one or more calculations on the converted data to produce oneor more outputs, wherein said performing one or more calculations areperformed according to instructions instantiated as forms; selectingfrom said one or more outputs to populate a final report; wherein theone or more outputs are standardized and are directly comparable tooutputs resultant from said method carried out on another set of one ormore other analytical instruments, irrespective of manufacturer or modelof the other analytical instruments; and outputting at least one of: atleast one of said outputs and said final report.
 2. The method of claim1, further comprising inputting the data to be converted from a nativecontrolling software of the analytical instrument from which data is tobe converted.
 3. The method of claim 1, wherein said performing one ormore calculations comprises data reduction, said data reduction beingcarried out by a data reduction engine, wherein the same data reductionengine may be used for data received from multiple analyticalinstruments.
 4. The method of claim 3, further comprising inputtingresults of said data reduction to a calculation engine and performing atleast one further calculation based upon said inputted results.
 5. Themethod of claim 1, wherein said forms contain data generated from atleast one of interactive manual input, information detected by a systemperforming said method and software of system, a computerized datasystem of an instrument from which data is being converted andcalculated reduced data.
 6. The method of claim 1, wherein said formsinclude launch points for executables that perform functions performedby said method.
 7. The method of claim 1, further comprising storingsaid outputs of performed calculations on the forms.
 8. The method ofclaim 7, wherein said selecting comprises selecting outputs from saidforms.
 9. The method of claim 7, further comprising storing said formsin a database as a repository of processed data.
 10. The method of claim9, further comprising identifying an audit trail based upon forms anddata stored in the database.
 11. The method of claim 1, furthercomprising identifying the hardware from which data to be converted wasoutputted from, based upon the rules-based algorithm associated with theform.
 12. The method of claim 1, wherein said calculations are performedto answer a series of questions relating to one or more performancetests designed to determine compliance of the analytical instrument orsoftware under consideration with a set of predefined criteria.
 13. Themethod of claim 12, wherein the set of predefined criteria comprisecriteria defined for regulated industries.
 14. The method of claim 13,wherein said predefined criteria comprise regulations set forth in theFood, Drug and Cosmetic Act.
 15. The method of claim 1, furthercomprising automatically storing results of one or more of thecalculations on the forms.
 16. The method of claim 15, furthercomprising manually inputting information on the forms.
 17. The methodof claim 16, wherein said manually inputting is performed interactivelythrough a user interface.
 18. The method of claim 1 wherein saidperforming one or more calculations comprises creating atechnology-independent, reduced metadata set from saidtechnology-neutral data.
 19. The method of claim 18, further comprisingcorrelating the metadata into a cohesive, audit-ready report thatfollows best practices for compliance documentation.
 20. A method ofcompliance testing at least one of analytical instrumentation andsoftware for analytical instrumentation, said method comprising thesteps of: displaying a test protocol on a user interface and prompting auser to input information regarding results of a test; prompting atleast one analytical instrument or software associated with ananalytical instrument to initiate the test protocol in response to aninput by the user into the test protocol displayed on the userinterface, or results from another analytical instrument in response toa test protocol run on the another analytical instrument; automaticallycalculating results of the test protocol run on the at least oneanalytical instrument, wherein said automatically calculating stepcomprises: converting data outputted from the at least one analyticalinstrument or software associated with an analytical instrument to atechnology-neutral format, which is independent of instrument type,instrument model, instrument manufacturer and data type of theanalytical instrument or software from which the data was outputted, andperforming one or more calculations on the converted data to produce oneor more outputs; and performing at least one of storing the calculatedresults for later use in an output report and outputting the calculatedresults to the user interface.
 21. The method of claim 20, furthercomprising selecting from said one or more outputs to populate a finalreport; wherein the one or more outputs are standardized and aredirectly comparable to outputs resultant from said method carried out onanother set of one or more other instruments and/or software,irrespective of manufacturer or model of the other analyticalinstruments.
 22. A system for standardizing characterizations of atleast one of analytical hardware and controlling software duringcompliance testing, said system comprising: algorithms for convertingdata from a native format as outputted by an analytical or otherinstrument to a technology-neutral format; a data reduction engineconfigured to reduce said data to a technology-independent, reducedmetadata set; a calculation engine configured to perform at least onecalculation on at least one of said data converted to atechnology-neutral format and said reduced metadata set to produce oneor more outputs required for a set of predefined criteria, wherein saidone or more outputs are produced to answer one or more questionsrelating to one or more performance tests designed to determinecompliance of the at least one of analytical instrument and softwareunder consideration with said set of predefined criteria; and a userinterface for displaying said outputs thereon.
 23. The system of claim22, further comprising an automatic detection engin0e configured todetermine at least one of instrument and controlling software specificinformation to automatically characterize said at least one of saidinstrument and said controlling software.
 24. The system of claim 23,wherein said automatic detection engine automatically performs at leastone of identification of said instrument or software and extraction ofperformance data from said instrument or software.
 25. The system ofclaim 22, further comprising interactive forms providing proceduralinformation including calculation instructions, and wherein said one ormore outputs are stored on said forms.
 26. The system of claim 25,further comprising means for mining said forms to extract metadataneeded to produce a final report.
 27. The system of claim 26, furthercomprising means for compiling the extracted metadata into the finalreport.
 28. The system of claim 25, further comprising a user interfaceconfigured to facilitate manual input to at least one of saidinteractive forms by a user.
 29. The system of claim 22, furthercomprising an enterprise content manager configured to provide a secureplatform and to manage data storage, data conversion, metadataextraction and archival of data.
 30. The system of claim 29, furthercomprising a business process manager configured to provide at least oneof e-mail notification, scheduling and review, and electronic signaturefunctions regarding at least one of said forms and said final reports.31. The system of claim 22, wherein said system accepts as input saiddata in the native format from native software that controls theanalytical instrument from which said data was outputted.
 32. The systemof claim 22, wherein said technology-neutral, standardized formatcomprises AnIML (Analytical Information Markup Language).
 33. A computerreadable medium carrying one or more sequences of instructions forcompliance testing of at least one of instrumentation and software,wherein execution of one or more sequences of instructions by one ormore processors causes the one or more processors to perform the stepsof: converting data outputted from at least one analytical instrument orsoftware to a technology-neutral, standardized format, wherein saidtechnology-neutral standardized format is independent of instrumenttype, instrument model, instrument manufacturer and data type of theanalytical instrument or software from which the data was outputted;performing one or more calculations on the converted data to produce oneor more outputs wherein said performing one or more calculations areperformed according to instructions instantiated as forms; selectingfrom said one or more outputs to populate a final report; wherein theone or more outputs are standardized and are directly comparable tooutputs resultant from said method carried out on another set of one ormore other analytical instruments, irrespective of manufacturer or modelof the other analytical instruments; and outputting at least one of: atleast one of said outputs and said final report.